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We consider the linear model 

Yi =x% + ei,i = 1,2,-■■ ,n, (1) 

where Xi are p x 1 known design vectors, /3o is a p x 1 nnknown vector of regression 
coefficients and {cj} are error variables. The M-estimate of f3o is dehned by minimizing 

n 

( 2 ) 

i=l 

where p is a convex fnnction. Important examples include Huber’s estimate with p{x) = 
{x‘^I{\x\ < c))/2-|-(c|a:| —c^/2)/(|x| > c), c > 0, where I{A) is the indicator function of the 
set A, the =2”^ regression estimate with p{x) = \x\^, 1 < <? < 2, and regression quantiles 
with p{x) = Pa{x) = ax'^ + {l — a){—x)^, 0 < a < 1, where x'^ = max(x, 0). In particular, 
if g = 1 or a = 1/2, then the minimizer of ([2]) is called the least absolute deviation 
estimate. There is a substantial amount of work concerning asymptotic properties of 
M-estimates. 
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We assume that p is a non-monotonic convex function on M with right and left deriva¬ 
tives 'ip+ cind xjj-. Choose 'ip such that 'ip-{u) < 'ip{u) < 'ip+iu) for all u G M. Write 
Sn = dn = niaxi<j<„and assume that S'„p > 0 for some integer uq 

and that n >nQ. 

Zhao (2002) established the following result. 

Theorem A. Assume that {ei,i > 1} is a sequence of independent and identically dis¬ 
tributed (i.i.d.) random variables. Suppose there exist positive constants A, Cq, Ci and 
6 G (0,1] such that the following conditions are satished: 

ip{u + h) — ip{u) < Cq for h G (0, A) and u G M, (3) 

Eip{ei) = 0, |E-?/^(ei-|- m)| > Cilul for |m| < A, (4) 

dn = 0{n~^). 

Assume further that E|'^(ei)|^/^ <ooif0<5<l and E|'0(ei)|^ < oo for some g > 1 if 
5 = 1. Then fdn /do a.s as n ^ oo. 

We will further discuss the strong consistency of (dn and obtain the following result. 

Theorem 1. In model ([I]), assume that {ej,z > 1} is a sequence of i.i.d. random 
variables. Suppose that conditions ([3]) and (0]) are satished and 

dn = 0{n~^). 

Then (dn —)■ (do a.s as n ^ oo. 

Proof For the technical proof, see the Appendix. 

Remark. In Theorem A, E|'0(ei)|'^ < cxo for some g > 1 is needed for the case 5 = 1. 
Theorem 1 is obtained without the condition E|'^(ei)|'^ < oo for some g > 1 in the case 
5 = 1. Theorem 1 improves the result of Theorem A. 
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Appendix 

To prove the main results of the paper, we need the following lemmas. 

Lemma 1 (c.f. Bennett, 1962). Assume that > 1} is a sequence of independent 

random variables such that EX„ = 0 and |X„| < b for all n > 1 and some 6 > 0. Denote 
= X]r=i Then for all e > 0, 





< 2 exp 


2he + 2Bl 


Lemma 2 (c.f. Choi and Sung, 1987). Let {X„,n > 1} be a sequence of i.i.d. random 
variables with EXi = 0. Assume that {an*, 1 < i < n, n > 1} is an array of constants. If 
maxi<j<„ |a„i| = 0(n"^), then 


ttniXi —)■ 0 as n —)■ cxD. 

i=l 

Proof of Theorem 1. Let Xni = S'n (3^ and = Sn^^n- Then 

n n 

E XniXT = Ip, I \Xni\\‘^ =p, dn= maX I \Xr, 

< ^ 1 <''»<’r) 


2 = 1 


2=1 


|2 

^ni 11 5 


(A.l) 


where Jp is the p x p identity matrix, and 11 • 11 is the Euclidean norm on 
Model (II]) can be rewritten as 


Yi = xT[3no P ei, i = l,2,---,n 


(A.2) 


and 


- ^nJn) = min p{Yi - xYp). (A.3) 

i=l ^ i=l 

Without loss of generality, we assume that the true parameter /Sq = 0 in model ([T]), 
i.e., (3no = 0 in flA.2p . Denote the unit sphere U = {(3 : (3 G M^, ||/5|| = 1}. Let £ > 0 be 
any given constant. Without loss of generality, it can be assumed that 2C'2e < A. Dehne 

n 

DniP) = X^{p(ei - xY(3) - p(ei)}, ^ eW 
2 = 1 


and 


Dnien^^^j) = ^{p(ei - “ P(ei)} 

2 = 1 
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(A.4) 


Tt .'V ^ 

= ^/ {'0(ei + t) - V'(ei)}c?t + ^wA7V'(ei), 7 e ? 7 , 

i=l “'O j=l 

= : An(7) + -^2n(7)) 

where Wni = —Hence 

inf D„(£n^/^ 7 ) > inf An ( 7 ) + inf An (7) > inf An (7) - sup | An (7)1- 

■y&U 'yGU 'yGU 'yeU 7eC/ 

We can divide U into N parts, Ui, U 2 , ■ ■ ■, Un, such that the diameter of each part 
is less than n“^ and N < {2n^ + 1)^. Let Tj be the smallest close convex set covering Uj. 
For a hxed Tj, there are three cases as follows. 

i) > 0 for each j E Tj, then there exists a 7 *^ G Tj such that wT'jij = inf{tcL^ ; 

7 € T,}. 

ii) wT'y < 0 for each ■y E Tj, then there exists a ■jij E Tj such that vjT'jij = sup{wL^ ; 

7 e Tj}- 

iii) wT'y > 0 for some 7 E Tj, and vjT'y < 0 for some 7 E Tj, then there exists a 
7 ij E Tj such that wT'jij = 0. 

Write 


G(t) = E'ip{ei + t), 4/iA) = A(ei + t) - A(ei) - AA), tER. 


By the monotonicity of 'ip. 


inf An (7) > inf inf An (7) A 

7gc/ ^ ^ i<j<N'reTj ^ ^ i<t<A^ 


inf / 


> 


inf 


i'lij 


l<j<N 


G{t)dt< 1 


i=l >^0 


{A(ei + t) - 'ip{ei)}dt 


Er=i GA)dt y 


(A.5) 


Let 7 E Uj and ■jij E Tj. By (lA.ll) and the definition of Uj and Tj, for large enough n. 


i=\ i=l 


< 

< 

< 


n n 

^=1 i=l 

n 

I - l)'XniXU{-iij + 7) 

i=l 

n 

II 'To -7 nil Xni IP (II 7o -7 

i=l 

n 

+ 2 ) II Xni IP 

i=l 

3pn~^ < 1 / 2 , 


7 II) 
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combining (lA.lj) . we obtain that for 1 < j < iV, 

n n n 


i=l 


2=1 2=1 
|2 


= II 7 11^ -1/2 = 1/2. (A.6) 

By ([5]) and the selection of e, for large enough n and for i = 1, 2, • • • , u and j = 1, 2, • • • , A^, 
Wni'lijl = < ^28 || ||< C' 2 £(l + < 2028 < A. 

It follows that by (jl]) and flA.bp . 


inf 




G(t)dt > inf CiV 




tdt 


i=i -^0 


c-i 


inf TT f 

l<j<N 2 ^ 


2 = 1 


> 


Ci£^n 


For I < j < N„ denote ^i{t)dt. By (j3]) and flA.bp . one has that 

|yi^')| < 2C'o|u;h7ul < 4 C'oC' 2£ =: Cg. 

For 1 < j < A, we also obtain that by 

_-^ / -N ^ / XL! '^J 

Y1 ^ + ^) “ '4’{ei))dty 


2=1 


2=1 




i'yij 


G{t)dt 




2=1 


7ij 


G{t)dt. 


i=l >^0 


Dehne event An by 


I AkTAhAh > i 

^ 'as" Er.i r-™ G(t)dt 2 

Noting the fact that N < {2n^ + 1)^ and using Lemma 1, we have that 

N 


n 1 ^ 

hi) - 1 


P{An) < >^E 

j=l i=l 


ilij 


G{t)dt 


i=i -^0 
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N 

< 2^ exp 
i=i 

N 




2Gs X i E”., r-™ G(i)* + 2Cs X 1E”., G(t)dt 

^ i-wL, 


2E2<'"p{-8C;^/, 


j=l i=lJ^ 

< 2(2n^ + l)^exp(—C 4 n), 

thus Yl^=iPi.^ri) < oo. By Borel-Cantelli lemma, it follows that P{An,i.o.) = 0. Hence 
with probability one for large enough n, 

sup - 

E”=, G(t)dt 2 

which implies by flA.Sp that with probability one for large enough n, 


inf An (7) > 

7 GC/ 


(A.7) 


Denote 



^ ^nil 


'^70 


^m2 


72 



,7 = 



^ ^nip j 


Wp / 


? ^ni — 


^nik 1 <C i <1 n 

-A ~ for hxed A; = 1, 2, • • • ,p. 

0 , * > n 


By flA.ip and dn = 0{n ^), for hxed k = 1,2, ■ ■ ■ , N, \xnik\ E|| Xni ||< dl/"^ < \fC 2 n 
and Er=i^n*fc = 1, thus 


< \/C^n ^ for n > 1. 


Using Lemma 2, it follows that 


n 


- 1/2 




a.s. 


2 = 1 


2=1 


Hence, 


n n 

n-^ sup |/2n(7) I = sup | V wA'^'tp{ei) \ = sup | V en^/^xP-i'ilj{ei) \ 
I&u -yeu “ 7ec/ ^ 

n p 


= en 




sup EE ^nik'yk'^ {,^i) 

i=l k=l 
P n 

sup ^ )7fc 


fc=l j=l 
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< en 


\E(E 

\ k=l i=l 


\ 11^1 

\ k=l 


= 


V n 

\ X] -^0 a.s., 

\ fc=i i=\ 


which implies that with probability one for large enough n, 


sup |/2n(7)l < 


7Gf/ 


16 


(A.S) 


Combining (1A.4I1 with (1A.7P and (lA.Sp . we have that with probability one for large enough 
n, 

inf Dn{en^^‘^-f) > ■ 

■yGU 16 

By the convexity of Dn{-), Dn{0) = 0 and the definition flA.Sp of (3*, it follow that 

inf > o| C {|| /3; ||< en^^^}. 

'y&U } 

Thus for any given e > 0, with probability one for large enough n, 

which implies that 


n 


- 1/2 


/?* ||—)■ 0 a.s. as n —)■ oo. 


Denote C(^) fhe smallest eigenvalue of a positive dehnite matrices A. Based on the 
result that if A and B are two positive dehnite matrices of order p, then 

tr(A5)>p(A)C(i?), 

where /i(A) is the largest eigenvalue of A (see Chen & Zhao, 1995). Now we take M = uq 
and n > no so that Sn > Sm > 0. By ([5]), one has that 

C(S„)(C(S«))-' < C(S„)tr(S-‘) < 

M 

= tT{S~^SM) = < Mdn < 


i=l 


and there exists a positive constant such that 1 < C 5 n~^^‘^({Sn‘^). It follows that 
II /3n II< II 11= || /3* ||— )■ 0 a.s. as n ^ oo. 

The proof of the theorem is completed. 
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